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ABSTRACT 

Langland and Baker introduced an approach to assess the impact of observations on the forecasts. In 
that approach, a state-space aspect of the forecast is defined and a procedure is derived ultimately relating 
changes in the aspect with changes in the observing system. Some features of the state-space approach are to 
be noted: the typical choice of forecast aspect is rather subjective and leads to incomplete assessment of the 
observing system, it requires availability of a verification state that is in practice correlated with the forecast, 
and it involves the adjoint operator of the entire data assimilation system and is thus constrained by the 
validity of this operator. This article revisits the topic of observation impacts from the perspective of esti- 
mation theory. An observation-space metric is used to allow inferring observation impact on the forecasts 
without the limitations just mentioned. Using differences of observation-minus-forecast residuals obtained 
from consecutive forecasts leads to the following advantages: (i) it suggests a rather natural choice of forecast 
aspect that directly links to the data assimilation procedure, (ii) it avoids introducing undesirable correlations 
in the forecast aspect since verification is done against the observations, and (iii) it does not involve linear- 
ization and use of adjoints. The observation-space approach has the additional advantage of being nearly cost 
free and very simple to implement. In its simplest form it reduces to evaluating the statistics of observation- 
minus -background and observation-minus-analysis residuals with traditional methods. Illustrations com- 
paring the approaches are given using the NASA Goddard Earth Observing System. 


1. Introduction 

Langland and Baker (2004, hereafter LB04) introduce 
a technique to examine the impact of observations on 
the short-range forecast. In that technique, an aspect 
of the forecast is defined and changes to the aspect are 
then associated to changes in the observing system. 
LB04 derive an expression to calculate observation im- 
pacts using the sensitivity (adjoint) operators of both the 
underlying forecast model and that of the analysis sys- 
tem. Errico (2007, hereafter E07) rederives the expres- 
sion of LB04, as well as introduces other expressions, 
using a Taylor series expansion of the forecast aspect 
when the initial conditions are subject to infinitesimal 
changes. Gelaro et al. (2007, hereafter GZE07) use an 
early version of the National Aeronautics and Space 
Administration (NASA) fifth-generation Goddard Earth 
Observing System (GEOS-5), together with the adjoint 
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of the GEOS-5 general circulation model (GCM), and 
the line-by-line adjoint of the gridpoint statistical in- 
terpolation (GSI) analysis system of Zhu and Gelaro 
(2008), to examine in detail the various approxima- 
tions in E07. More recently, Daescu and Todling (2009, 
hereafter DT09) show that the expressions in E07, and 
higher-order accurate ones, can also be derived using 
a parametric approach in conjunction with numerical 
quadrature methods. 

When the technique of LB04, E07, and DT09 is used 
to assess the impact of the observing system on the fore- 
cast one must keep in mind the assumptions and limita- 
tions of the method. First, the definition of the forecast 
aspect reflects whatever measure one judges to be rele- 
vant to describe features of interest. Much of the work 
done in this area has thus far focused on the impact of 
observations in the 24-h forecasts. The forecast aspect 
has typically been chosen to measure the 24-h forecast 
error, as defined by a linearized total energy norm. In 
addition, the works of LB04, GZE07, and DT09 have 
applied a projection operator to the forecast error mea- 
sure that intentionally excludes errors, roughly, above 
100 hPa. Other works have used very specific projection 
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operators as they are aimed at evaluating targeted ob- 
servations (Langland 2005; Buizza et al. 2007; Cardinali 
et al. 2007; Kelly et al. 2007; Rabier et al. 2008). Un- 
deniably, use of such measures restrict conclusions drawn 
about the observing system and its impact on forecasts 
in general. 

Second, a further complication associated with the 
definition of typical forecast aspects is the need to use 
a verifying state. It is common practice to evaluate the 
quality of forecasts with respect to underlying analyses, 
as for example when calculating forecast skill scores. 
However, one must recognize the limitations of such a 
practice since correlations between the forecasts and 
the verifying analyses introduce unknown features to the 
results, particularly when relatively short-range forecasts 
are involved. 

Third, the approximate observation impact formulas 
of LB04, E07, and DT09, are limited by the validity of the 
adjoint models they invoke. In particular, the validity of 
the underlying general circulation model adjoint restricts 
the studies of observation impact to those related to 
short-range forecasts only , though there are currently 
attempts to extend the validity of adjoints by defining 
optimal trajectories (see Barkmeijer and Stappers 2011). 
Lack of proper representation of full nonlinear pro- 
cesses introduces yet another uncertainty factor (e.g., 
some adjoints have poor or no representation of con- 
vective processes). Furthermore, the nonlinearity of many 
analysis systems (Tremolet 2007, 2008) introduces an- 
other layer of complexity and limitation to the adjoint- 
based approach. The ability to use adjoint-free diagnostics 
is advantageous, at least from the practical point of 
view. Liu and Kalnay (2008) have recently introduced 
an ensemble-based observation impact technique that, 
by construction, avoids the need for adjoints (see also 
Liu et al. 2009). Still, the work of Liu and Kalnay is 
based on a rather subjective forecast aspect, and con- 
tinues to rely on the availability of verification states. 

More generally, it is not necessarily clear how using 
information derived from observation impact evalua- 
tions can aid the development and enhancement of data 
assimilation systems. Since observation impact results 
tangle the use of observations through the data assim- 
ilation process with errors in the short-range forecast 
that are not part of the data assimilation process, it is 
hard to disentangle the information provided by ob- 
servation impact studies to make sound decisions about 
the use of observations. It is thus the main point of 
the present work to argue that one might as well stick 
with traditional methods of evaluating the statistics 
of observation-minus-background (OMB) and, to some 
extent, observation-minus-analysis (OMA) residuals to 
allow one to infer the required information to improve 


upon the cycling data assimilation scheme and the use 
of observations (e.g., Hollingsworth and Lonnberg 1989; 
Daley 1992; Dee and da Silva 1999; Desroziers et al. 
2005a, b; Chapnik et al. 2006; Lupu et al. 2011). 

To build toward the main goal here, the present work 
examines the impact of observations on the forecasts by 
introducing measures based on weighted observation- 
minus-forecast (OMF) residuals directly. The weighted 
difference of the squared OMF residuals calculated 
from two forecasts issued from two consecutive analyses 
serves as a means to evaluate the impact of observa- 
tions on forecasts. The limiting case in this approach is 
one in which the forecast is reduced to be the very 
short-range background, thus leading back to the more 
traditional residual-based techniques cited above. The 
advantages of this approach are: (i) it still allows for a 
complete assessment of the observing system; (ii) it 
avoids introducing undesirable corrections to the fore- 
cast aspect; (iii) it involves no approximations to the 
model and analysis operators, avoiding adjoints and 
consequently being devoid of their limitations; and fi- 
nally, (iv) it requires only straightforward calculations 
involving quantities readily available in most practical 
data assimilation systems. 

Arguing from the perspective of estimation theory, 
we examine the statistics of the various approaches to 
observation impact. It becomes evident that the state- 
space approach is, in principle, more encompassing than 
the observation-space approach. It is shown that, under 
idealized conditions, it is possible to choose a state- 
space measure that obtains the same expected obser- 
vation impact as that obtained with a corresponding 
observation-space measure. It is further shown that op- 
timality of the data assimilation system implies that the 
expected observation impacts are always negative in the 
expected mean sense, that is, assimilation of observa- 
tions always amounts to improvements in the forecast. 
The analytic evaluation then goes on to examine the 
consequences of having to replace the unknown truth 
with an arbitrary verifying state, and the consequences 
resulting from when the verifying state is chosen to be 
the analysis. At this point it becomes rather clear that 
the observation-space approach is preferable over the 
state-space approach since the former uses the obser- 
vations for verification and therefore does not introduce 
spurious correlation factors as does the latter. Ulti- 
mately, it is argued that if the objective of calculating 
observation impacts is to aid the underlying data assimi- 
lation procedure and help decide how best to use the 
available observations, employment of the observation- 
space approach, with the additional simplification of tak- 
ing the fields involved in the measures difference to be 
the analysis and the background, suffices. 
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Illustration of the various points raised along the text 
is given by using GEOS-5. The dependency of the ap- 
proaches on the definition of the forecast aspect is given 
by calculating observation impacts on the 24-h forecasts 
using three distinct measures based on (i) the tropo- 
spheric, linearized total energy norm used in LB04 and 
other works; (ii) a modified version of the linearized 
total energy norm that more evenly weighs the vertical 
(e.g., Errico et al. 2007); and (iii) the inverse of the ob- 
servation error covariances used by the analysis system, 
projected onto state space. There are important differ- 
ences among results obtained with these different mea- 
sures. Though most differences can be well understood, 
the clear message is that the ranking of the observing 
system that almost naturally follows from these studies is 
a rather subjective matter. The theoretical consider- 
ations of section 2 suggest that the observation-space 
approach can be used in practice to try to assess the 
consequences of verifying forecasts against analyses. The 
experiments with GEOS-5 indicate these verifications 
tend to overestimate the impact of the observations. 
This is even more so when the state-space approach is 
used. Last, given the implicit subjectivity of the various 
measures of observation impacts some illustration is 
given to enforce the view that one might as well rely on 
the, readily available, usual OMB and OMA residuals 
to try to assess how observations are used in the as- 
similation process and what can be done with this in- 
formation to make system improvements. 

In what follows, section 2 compares the state- and 
observation-space approaches from the perspective of 
estimation theory. Section 3 shows results comparing 
observation impacts on the forecasts obtained using var- 
ious measures and linking to the results of section 2. 
Closing remarks appear in section 4. 

2. Measures of observation impact 

a. Background 

Let us write the expression describing a forecasting 
model as 

X k\k-m+l ~ m k,k-m + l( X k-m + l\k-m + l) > ( 1 ) 

where here, borrowing from the notation of estimation 
theory (e.g., see Cohn et al. 1994), the /^-vector forecast 
state x£|£_ m+1 , at time t k , is derived by integration of 
the model m, from time t k ~ m+ 1 to time t k , starting from 
an analysis state x a k _ m+1 \ k _ m+1 calculated at time t k - m+1 . 
The subscript notation i \ j indicates that the estimate at 
time ti is obtained by using observations up to and in- 
cluding observations at time tj , for j < i. In the linear 


case, this notation is associated with the more profound 
statement that the minimum variance estimate is the 
conditional mean (e.g., Cohn 1997). In the nonlinear 
case considered here caution must be exercised with 
this interpretation, but the notation is still informative. 

In data assimilation, the analysis is an estimate that 
combines a model background field, 

x k-m+l\k-m ~ X k-m+l\k~m ~ m k-m+l,k-m( X k-m\k-m) > 

( 2 ) 

with the p^_ m+ i-vector of observations y k _ m+1 , at time 
tk-m+i, and can conveniently be written in the follow- 
ing form: 

= ^b I ic 

x k—m+l\k—m+l x k—m+l\k—m ' IK k—m+l\k—m 

^ [y&— m+1 ^k— m + l( X k— m + l\k— m)] ’ 

( 3 ) 

where h k - m +i is the p k - m+1 observation operator that 
transforms model states into observables: K k - m+1 \ k - m is 
a general, not necessarily optimal, n X p k _ m+1 matrix of 
weights used to update the background state given ob- 
servations at time t k - m+1 ; and d k - m+1 \ k - m = y£_ m+1 - 
b*-m+i(4-m+i|*-m) is the Pk-m+ l-vector of residuals 
representing the difference between the actual and 
the model-predicted observations. The analysis expres- 
sion (3) is representative of a three-dimensional varia- 
tional formulation; for convenience, it is written here 
at time t k - m+1 rather than at the usual time t k ; taking 
(k — m + 1) — > k converts it into its familiar form. 

Studies of observation impact on the forecast in- 
troduce a scalar functional to measure the quality of 
the forecast and establish a common means to evaluate 
the effect of observations on the forecast. A typical 
quadratic scalar measure takes the following form: 

e k\e = ( x k\e - 4) T U( x i - 4) • ( 4 ) 

where the n X n symmetric positive semidefinite matrix 
stands for the weight given to the forecast error 
e k\£ = X l\£ ~ x 5c> evaluated at time t k , for i < k. In practice, 
one cannot calculate the weighted forecast error in (4) 
since it involves the unknown true state x k . Instead, 
a verification state x k is used, so that the forecast error 
becomes e v k ^ = - x k , and the scalar measure (4) is 

replaced with 

e l\e = (4\e - 4) XT fc( x i - 4) • (4 

Naturally, the verification state is usually taken to be 
an analysis, the consequences of which are discussed in 
what follows. 
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The impact of observations on the forecast can be 
evaluated by studying how the forecast error measure 
(5) changes with respect to changes in the initial condi- 
tion in (1). These changes can be thought to be a result 
of changes due to the assimilation of observations. As- 
suming the changes to be infinitesimal, E07 and DT09 
derive various approximations to the corresponding 
change in the forecast error measure. Relevant to the 
present work are the following first- and second-order 
(per DT09) accurate formulas: 


X- = 


8eT = d 


.K 


[ k—m+l\k—m r ^k—m+l\k—m y x bC k\k-m' 


\„ h el 


(6a) 


V V,2 jT Qr T 

0e k ~ 2 a ^~ m+ M lc ~ m K- m +l\k-m 

^ \y x b ^k\k— m ^x a ^k\k— m+l] ’ 


(6b) 


where the gradient vectors above can be written as 

V* eV k\e = m lk,k-m+ J k ^k\f: - > ( 7 ) 


with the n X n matrix, 


M 


g;k,k— m+l 


_ dm k,k-m + lW 


dx 


( 8 ) 


standing for the Jacobian of the model m in (1) in- 
tegrated from time t k - m+1 to t k , and linearized about 
integrations started from either the background x § = 
m(x*_ m+1|(t _ m ), indexed g = b, or the analysis \ g = 
m ( x k- m +i\k-m+i)’ indexed g = a, respectively; in (7), 
£ = k — m or £ = k — m + l, depending whether the 
error (5) is evaluated for a forecast started from a back- 
ground or an analysis, respectively. Equation (6b) first 
appeared in LB04. Figure 1 gives a schematic represen- 
tation of the various relevant times involved in the cal- 
culation of the formulas above. 

A proxy for the infinitesimal error change 8e v k of the 
forecast error measure (5) is simply the difference be- 
tween the error measure calculated for forecasts issued 
from two consecutive analyses: 


^ k ^k\k~m + l 


— e 


V 

k\k—m ' 


( 9 ) 


This proxy is used, for example, in the work of Liu and 
Kalnay (2008) to derive an expression for observation 
impact that can be more readily calculated in the con- 
text of ensemble data assimilation procedures. Simi- 
larly, the present work relies on the argument that much 
of what is obtained with approximations such as those in 
(6) can be obtained using differences such as in (9). In- 
deed, in many respects the error difference (9) is more 



Fig. 1. Schematic representation of time line and the relevant 
forecast error definitions. 


revealing than its infinitesimal counterpart exactly be- 
cause the former is capable of describing what finite-size 
changes in the initial conditions, due to the assimilation 
of observations, imply to the forecast. 

The approximations in (6) express the change in 
forecast error in the form of an inner product be- 
tween a “sensitivity” vector and the observation-minus- 
background residual vector, Consequently, 

it is possible to break down the inner product into its 
many elements — corresponding to the contributions 
from the various components of the observing system 
to the error change. These individual contributions de- 
fine the corresponding individual observation impacts. 

b. A simple alternative forecast error measure 

In observation-space, we replace the state-space fore- 
cast error measure (5) with the observation-space fore- 
cast error defined as the weighted difference between 
the model-predicted observations h^(x^) and the ob- 
servation vector y£, at time t k , for £ < k, that is, 



( 10 ) 


where C k is a p k X p k positive semidefinite suitable 
weighting matrix, and the forecast error is calculated 
for a forecast started at time t £ < t k . The first equality in 
the expression above is written to emphasize the anal- 
ogy between this error expression and that in (4) or (5), 
suggesting that now the verification state is simply re- 
placed with the observations. 

The observation-space counterpart of the error dif- 
ference in (9) is now 

8e k = e k\k-m+l ~ e k\k-m ' (H) 

The error-change 8e y k is a scalar that can be calculated 
for any portion of the observing system. Error differ- 
ences such as (9) and (11) identify the improvement (or 
degradation) in the prediction at time t k from a forecast 
started at time t k - m+1 over that started at time t k - m . In 
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other words, these differences are a measure of the im- 
pact that assimilating observations at time t k - m has on 
the forecast at time t k . Partitioning the impacts into in- 
dividual contributions from the various components of 
the observing system is straightforward since (10) and 
(11) are calculated directly in observation space. On the 
other hand, when the error measure is defined in state 
space, as in (5), one must use approximate expressions 
such as those in (6) to be able to partition the impacts as 
desired. As long as the verifying observations amount 
to a robust set, such as being relatively homogenous in 
time, one should be able to derive statistically reliable 
observation impacts using the observation-space ap- 
proach. The same comment applies to observation im- 
pacts derived with the state-space approach. 

However, in the observation-space measure the ob- 
servations are taken at verification time (by construc- 
tion), rather than at initial time, as in the state-space 
measure. It is the view of the present work that, as long 
as the observing system is relatively homogenous when 
comparing the initial and verification times, this is not 
a serious issue since any statement based on residuals 
can only be made on the basis of statistics; the statistics 
must be robust for there to be any confidence in the 
results. For typical evaluations of, say, observation im- 
pact on the 24-h forecasts the verification times are 
aligned with the initial times and no issues are at stake; 
if impacts on the 18-h forecasts were to be calculated in- 
stead, the observing network differences between initial 
and verification time would be something to watch for. 

A relationship between the error changes (6) and (11) 
can easily be established. If one expands the forecast 
error vectors in (10) following similar arguments to those 
used to derive (6), the residual-based forecast error 
change (11) can be approximated to first- and second- 
order accuracy as 


delude 


y, i 

k 


-d 


T 

k—m+l\k—m 


k T 

rv /c— m + l \k—m 


¥ x b e 


y 

k\k—m 


(12a) 




-8e 


y,2_ 


1 


K 


2 u k—m+l\k—m rK k—m + l\k—m 
X[^^_ m + V xae ^_ m+1 ], (12b) 


respectively, where the gradients are calculated as 


^xs e k\£ 


y - 


^^\k,k-m+l^k\S'k^k\£ ’ 


(13) 


for H k \i representing the p k X n Jacobian matrix, 
_ <%( x ) 


H 


'k\£ 


dx 


(14) 


k\i 


of the nonlinear observation operator linearized 
about the state x 8 k ^, for i = k - m + 1 when g = b, and 
£ = k — m when g = a. Higher-order expressions can 
be derived following the procedures in E07 or DT09. 
The approximations in (12) are useful to highlight the 
fact that both the state-space and observation-space ap- 
proaches refer to the same initial perturbation to the 
forecast, that is, the transpose of the analysis increment 
vector bx a k ^ m+Mk _ m = K k - m +i\ k - m & k - m +i\ k-m- When us- 
ing (11) to calculate observation impacts this initial 
perturbation appears only implicitly. Indeed, the dif- 
ference being these approximations comes from the 
gradient vector representing how initial perturbations 
propagate within the time window starting at t k - m and 
ending at the verification time t k . For example, in the 
first-order approximation in (6a) the gradient vector 
is \ x be v k \ ( = 2M^. kk _ m+ {T k e v kl( , whereas in (12a) the gra- 
dient vector V x( ,e^=- 2M^_ m+1 H^C*d*|*. This is 
simply a consequence of the choice of error measures; 
the negative sign appears simply because d k \£ is a re- 
sidual defined as observation-minus-field instead of 
a typical error defined as field-minus-observation. 

c. Relationship between the state- and 

observation-space error measures 

To more accurately establish a relationship between 
the state-space- and observation-space-based error mea- 
sures we consider now the definition of the error mea- 
sures introduced above in a probabilistic sense by referring 
to their expected values. This goes along with the think- 
ing that any statement made in data assimilation must 
have a statistical basis. For the sake of argument, we 
simplify matters by considering the linear case, when 
the matrices M g; ^, K k ^ and H k \ £ become state inde- 
pendent and can be written more compactly as M^_ m , 

and H k , respectively. Taking the linear version of the 
analysis equation (3) at time t k - m and substituting into 
the linear version of (2), the forecast error of a forecast 
issued from the analysis at time t a = t k - m+ 1 , and valid at 
time t = t k , can be expressed as 

J _ J _i_ |\/| it H 

t k\k—m+l t k\k~m *'k,k— m+l a :— m+l u k— m+l\k— m ’ 

(15) 

which also holds to first-order accuracy for the non- 
linear case. The expression relates error in the forecasts 
valid at the same time, but issued from analyses one 
cycle apart. Since the only difference between forecasts 
issued from two consecutive analyses, at times t b = t k - m 
and t a = t k - m+1 , is the observations assimilated in the 
more recent analysis, the error in the forecasts differ by 
how much the incremental difference introduced by the 


May 2013 


TODLING 


1489 


assimilated data propagates in time, that is, by how the 
term 8x a k _ m+1 \ k _ m evolves under the dynamics M^_ m+1 . 

Taking the expected mean of the true state-space 
measure introduced in (4) gives 

^k\k—m) (( e k\k— m) k^k\k—m)^ 

= Tr{T,[((^_ m )(4 | ,_ m ) T )]} 

= Tr{T,P{ | ,_ m }, (16) 

where P{ |yt _ m - (e f k[k _ m (e f k] k _J T ) is the forecast error 
covariance matrix, with ( • ) representing the expectation 
operator, Tr( ) stands for the trace operator, and we 
used the trace property Tr(A T TB) = Tr(TBA x ) for ar- 
bitrary matrices A and B of dimension nX p. Therefore, 
the expected mean of the forecast error change calcu- 
lated for forecasts from two consecutive analyses is 

<«**> = Tr{T,AP{}, (17) 

where AP{ = P^ k _ m+1 ~ P f k \ k - m is the difference between 
the two forecast error covariances corresponding to the 
two lagged forecasts. 

As suggested above, the impact of observations can 
also be evaluated by examining the expectation of the 
observation-space measure, that is, 

^ k\k—m ) ((^k\k—m) ^k^k\k—m)^ 

= Tr{C,[((d^_ m )(d^_ m ) T )]} 

= Tr{C,r^_ m }, (18) 

where we introduce the p k Xp k residual error covariance 
matrix: 

r £| k-m k\k-m^k\k-m ) = + R k‘ ( 19 ) 

Using this observation-space measure the change in 

forecast error due to the assimilation of observations 
becomes 


with no assumptions made on optimality and model 
error. Furthermore, recall that at any time t k , the anal- 
ysis error covariance can be written as [e.g., see Cohn 
et al. (1994), their Eq. (2.33)]: 

P ^ = ('- WP{|*-i+AP£| fc , (22) 

where the increment matrix AP^ = AK^r^AK x in- 
corporates all the suboptimality in the analysis error 
covariance, with \K k = K k - K k being the difference be- 
tween a general gain matrix and the optimal Kalman 
gain matrix K /o with 

K* = p£|*-iH In 1 - (23) 

From (22) it follows that, 

p *r p i-i = -Wl + ^. (24) 

is negative semidefinite in the optimal case, when 
AF * a k \k = 0, and therefore so is the forecast error co- 
variance difference in (21), AP{ < 0. Applying this re- 
sult at time t k - m+1 and combining it with (17) and (21) 
shows that in the optimal case the expected forecast 
error change (8e k ) is guaranteed to be nonpositive, 

^ e k ;) = ~ r ^ r {^k^k,k-m+l^k-m+l^k-m+l 

X ^k-m+l^k,k-m+l} ~ ( 2 ^) 

since the kernel inside the trace operator is positive 
semidefinite, with equality holding when T k = 0. In the 
optimal case, and in the expected mean sense, assimi- 
lation of observations is guaranteed to reduce forecast 
errors. 

Similarly, since in the optimal case AP{ < 0, the 
observation-space-expected forecast error change in 
(20) is also nonpositive, 

(K> 0|t -Tr{HlC t H t M w _ m+1 K t . m+i r t _ m+1 

rp rp OOt 

xKj_ m+1 Mj,,_ m+1 }<0, (26) 


<^> = Tr{H*C fc H,AP;[}, 


(20) 


for Se y k =8e y klk _ m+1 -8e y k{k _ m . 


After some algebra, the difference between the two 
forecast error covariances defining AP^ is shown in ap- 
pendix A to be 


k ^k,k-m+l^k-m+l\k-m+l 




k— m+l\k— m)^k,k— m+1 ’ 

( 21 ) 


corroborating again that, in the expected mean sense, 
assimilation of observations leads to forecast error re- 
duction. 

A particular choice of weighting matrix T k , namely 
T k = H X C^H^, leads to 


<^(T, = T,)>°U^)> (27) 

that is, in the linear optimal case one can choose the 
forecast error measure in state space to obtain the same 
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total expected error as that obtained in observation 
space. Conversely, since the observation space is nor- 
mally smaller than the state space, one cannot choose 
C k to obtain the same total expected forecast error as 
obtained in state space. 

d. The role of the verification 

One can also inquire about the relationship between 
the change in the true expected forecast error (8e k ) and 
that defined with respect to a verification state ( 8e v k ). 
Sticking with the linear, but not necessarily optimal, case 
we show in appendix B that the relationship between 
these expected errors can be written as 

M> = (Se k )~ 2Tr[Kj_ m+1 ML- m+ iT Jt (^dJ_ m+1 | fc - m )], 

(28) 


where e k = x k — x k is the true error in the verification 
state. Replacement of the truth with a verification state 
results in a term involving the correlation between the 
error in the verification and the background residuals. 

More specifically, when the verification state is taken 
to be the analysis, x v k = x^, a typical choice in practical 
applications, the expression above can be worked out 
in more detail to give 


8d 


* =<««*> -2Tr 


^k-m + 1 ^k,k- m + 1 ~^k 


^ ^^k,k—m+l^^k—m+l^'k—m+l 

m-2 ~ \ 

^ ^k,k-j^k-j^k-j\k-j-l^k-m+l\k-m) ) * 

7=0 J\ 

(29) 

This relates the perceived error change, calculated us- 
ing the analysis for verification, with the actual error 
change. To arrive at this result (see appendix C) one 
requires typical assumptions of linear filtering theory 
(e.g., Jazwinski 1970, chapter 7): that model errors be 
uncorrelated with observation errors, and that forecast 
errors be uncorrelated with observation errors for all 
times larger than the time the forecast begins. Two terms 
prevent the perceived error change ( 8e v k =a ) from equal- 
ing the actual error change. One is the first term in the 
trace expression involving the difference AK k - m+1 be- 
tween the suboptimal and optimal gains. The other is 
the second term in the trace expression involving the 
cross covariances of the various observation-minus- 
background residuals between the analysis time t k - m+2 
and all times up to the verification time t k . It is only in 
the optimal case that both these terms vanish: the first, 
for obvious reasons, AK^ = 0; the second, because the 


sequence of observation-minus-background residuals , 
becomes the sequence of innovations , which 
is white in time, and with all the time cross covariances 
becoming zero 1 (Kailath 1968; Daley 1992; see also 
Anderson and Moore 1979, their section 5.3). There- 
fore, in the optimal case and in the expected mean sense , 
verifying against the analysis is the same as verifying 
against the truth when it comes to evaluating the ex- 
pected forecast error change under consideration. 

Of more practical interest is the question of what 
happens when, using the observation-space measure (10), 
the observation vector is replaced with a verification 
state x v k projected onto the observation space by the 
observation operator, that is, when y k is replaced with 
h k (x v k ), or in the linear case, H^x^. Following similar 
steps to those taken in appendix B to derive (28), but 
now applying the rational to the observation-space mea- 
sure, it can be shown that the observation-space error 
measure change takes the following form: 

(8eD = K> -2Tr[kJ_ m+1 ML- m+ iHjC,H, 

X(44- m+ i\k -«)]. ( 30 ) 


when a verification state replaces the observations. As 
before, the perceived expected observation error change 
(8e y k ~ v ) differs from the actual expected error change 
(8e y k ) by a similar term to that found when examining 
this relationship in state space, that is, the cross correla- 
tion between the verification error and the observation- 
minus-background error. Analogously, as one might 
expect, when the verification is taken to be the under- 
lying analysis, this relationship becomes 


(8e y = a ) = (8e y k )-2Tr 


K-m+l M k,k-m+l H k C k H k 


^ ^^k,k— m+l^^k— m+\^ k— m+1 


m—2 

X ^k,k-j^k-j^k-j\k-j-l^k-m+l\k-m) 
7=0 


(31) 


(this can be derived following the rational of appendix 
C). Analogously to (29), equality here only holds in 
the optimal case, when AK^ = 0 and the sequence of 
observation-minus-background residuals becomes the in- 
novation sequence. Unlike the expressions (28) and (29), 


1 Note that the cross variances, the cross terms calculated for the 
same time, are not zero, but they also do not appear inside the 
summation sign. 
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the observation-space expressions (30) and (31) can, in 
principle, be verified in practice. Still, one thing to notice 
related to expression (30) is that it assumes the verifying 
observations to be fully independent of the analyses. 
In many practical data assimilation applications this 
assumption is violated because of procedures to perform 
observation bias correction. 

e. General remarks 

Both the state- and observation-space measures con- 
sidered above involve what are in principle arbitrary 
weighting matrices T k and C k , respectively. One partic- 
ular choice has special meaning: when they become the 
inverse error covariance matrices of the corresponding 
underlying error. That is, 

02a) 

^k = ^k\k-tn^k\k-m^k\k-m + R J 1 • (32b) 

When T k is defined as in (32a), the forecast error (4) 
becomes a measure of the accuracy of the fit of the 
forecasts to the truth. In particular, when m = 1, the 
forecast error covariance becomes the background error 
covariance, B = p l\k~ v and the forecast error measure 
(4) becomes the background error term present in the 
cost function of variational data assimilation methods 
(e.g.: Courtier and Talagrand 1987; Lewis et al. 2006, 
their chapter 20). 

Similarly, when C k is defined as in (32b), the forecast 
error (10) becomes a measure of the accuracy of the fit 
of the model-predicted observations to the actual ob- 
servations. When m = 1, the weight matrix C k becomes 
the inverse of the familiar residual (“innovation”) error 


covariance matrix used in, say, observation-space-based 
analysis procedures (e.g., Cohn et al. 1998). In practice, 
this weighting factor can come from prescribed statistics 
or from an estimate obtained directly from sample error 
covariances constructed from the observation residuals 
themselves; similar to, say, the treatment of consistent or 
inconsistent error statistics of Lupu et al. (2011). 

The case m = 1 is important, and it is the only measure 
that can directly be associated with the data assimilation 
cycle. In this case, the observation-space measure in (10) 
reduces to 


= [KKik) - y*] Tc J h /i( x *i*) _ y °k\ 

- [ h *( x *i*-i) - y*] Tc J h fc( x *i*-i) - y *]. (33) 

where x l\k - 1 = x i\k- 1 and x i\k = K\k- Thus ’ the expression 
above is simply the difference between the weighted 
OMA squared residual with the corresponding OMB 
residual. This quantity is readily available in any analysis 
system. The particularly convenient choice, 

c * = r A (34) 

turns the difference above into the difference between 
the final and initial “ J Q ” term of the typical variational 
assimilation problem. 

Furthermore, simple algebra shows that the expected 
mean of the measure in (33), with weights from (34), 
relates to a version of the degrees of freedom for signal 
(DFS) diagnostic that uses residuals based on posterior 
estimates (analysis residuals). That is, 


(8e y k ) ~ -f k - ([h k (x a klk ) - hyt( x * l *- 1 )] T Rfc 1 K ~ h *( x *|*-i)]) 

w ~fk - (***1*- 1 (r^ii- 1 i P*|*_ i H i 1 )d yfc | /t _ i ) + 0( AK fc | jfc _ 1 ) 

°=~fk ~ ^w-Kk-iHlk-iKk 1 ), ( 35 ) 


where 

fk - ([ h k( x k\k ) - h fc ( x *i*-i)] TR * Hy* - h*(**i*)]> (36) 

is the DFS as derived in Lupu et al. (2011) [see Eq. (10) 
in that work]; the second approximate expression in 
(35) follows from noticing that h^(x^) - h/ < (x^_ 1 ) ~ 
H*|jfc-iKfc|*-i[y£ - Mxfcijfc-i)]; and the third equality fol- 
lows from applying the trace operator and invoking op- 
timality, when \K k \ k -i = 0. Thus, the observation-space 
measure for m = 1 is directly related to a DFS-like 


diagnostic for a suitable choice of C k . This is not very 
surprising: one should expect various diagnostics in- 
volving the same quantities (i.e., OMBs and OMAs), to 
provide similar information. Further insight is gained 
by examining the last equality, optimal expression, in 
(35). Since the second term on the right-hand side is 
positive, we see that when the overall impact under the 
observation-space measure indicates that a set of ob- 
servations degrades the forecasts, that is when (8e k ) is 
positive, the corresponding DFS will have a negative sign; 
observations with negative (8e k ), shown to contribute to 
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improve the forecast under the observation-space mea- 
sure, may have DFS of either sign. In other words, if the 
sign of the measure is used to define impact, both mea- 
sures agree when observations degrade the forecasts, but 
they might disagree on which observations contribute to 
improve forecasts. This apparent contradiction relates 
to the uncertainty associated in these statistical measures. 
If we accept both measures in question as reasonable 
ways of determining the impact of observations, then the 
degree to which their results can be trusted is measured 
by the second term on the right-hand side of (35). Es- 
sentially, the term Tr(H^|^_ 1 P^ A: _ 1 H^_ 1 R^ x ) provides 
an error bar for the reliability of impacts calculated with 
either method; impacts within the error bar can be con- 
sidered neutral. 

As hinted above, it is the view of this author that the 
only measures of observation impact that have direct 
implications to data assimilation are those calculated 
from (33) or, analogously, from (36). It is hard to see 
how general measures such as those in (9) and (11), for 
m > 1, can be used to aid the cycling of data assimilation 
procedures. The sensitivities of the forecasting model 
beyond the integration time window of a typical assim- 
ilation cycle do not participate in the formulation of data 
assimilation procedures. It is a basic fact of data assimi- 
lation, and generally estimation theory, that the statistics 
of the background- and analysis-observation residuals 
teach us what can be learned of the behavior of the data 
assimilation system and how it uses the observations. 
This is the reason why maximum likelihood procedures 
(e.g., Dee and da Silva 1999) and chi-square-based mea- 
sures (e.g., Menard et al. 2000) are solely based on these 
residuals (e.g., Desroziers et al. 2005b). This is not to say 
there is nothing to be learned from what has become 
known as “observation impacts”; it is just that it is un- 
clear how information obtained from such diagnostics 
relates to the cycling of data assimilation. Indeed, if we 
were to identify components of the observing system 
leading to particularly undesirable features to, say, the 
24-h forecasts and a decision was made to change the 
manner in which these components are handled during 
the assimilation cycle , it is certain that such changes 
would lead to unpredictable results in the cycle. In other 
words, one would not be able to tell in advance if the 
changes would lead to improvement or degradation to 
the quality of the assimilation cycle, much less to the 
quality of future 24-h forecasts. 

3. Illustrative results 

This section presents a brief illustration of the main 
points made in the previous section. GEOS-5 (Rienecker 
et al. 2008) is used for this purpose. GEOS-5 assimilates 


observations using the incremental analysis update tech- 
nique of Bloom et al. (1996). It consists of a global at- 
mospheric model developed at Goddard and an analysis 
system developed jointly by the National Oceanic and 
Atmospheric Administration (NOAA) National Cen- 
ters for Environmental Prediction (NCEP) and the 
NASA Global Modeling and Assimilation Office. The 
GEOS-5 GCM retains an updated version of the finite- 
volume hydrostatic dynamical core (Lin 2004) from its 
predecessor GEOS-4. The GEOS-5 GCM is built un- 
der the infrastructure of the Earth System Modeling 
Framework (Collins et al. 2005) used to couple together 
various physics packages including a modified version 
of the Relaxed Arakawa-Schubert convective param- 
eterization scheme of Moorthi and Suarez (1992), the 
catchment-based hydrological model of Koster et al. 
(2000), the multilayer snow model of Stieglitz et al. 
(2001), and the radiative transfer model of Chou and 
Suarez (1999). Furthermore, the GCM is accompanied 
by its adjoint model (ADM), which is essentially the 
ADM of the finite-volume dynamical core of GEOS-4 
(Giering et al. 2005), with added vertical diffusion and 
a polar filter (Errico et al. 2007). 

The GEOS-5 analysis component consists of the GSI 
system. The GSI implements a three-dimensional varia- 
tional data assimilation (3D-Var) using the incremental 
approach of Courtier et al. (1994) for minimization and 
the preconditioning strategy of Derber and Rosati (1989). 
The background error covariance is implemented as a 
series of recursive filters producing nearly Gaussian and 
isotropic correlation functions (Wu et al. 2002). Satellite 
radiances are processed using the Community Radiative 
Transfer Model (CRTM; Kleespies et al. 2004) and the 
online bias-correction procedure of Derber and Wu 
(1998). Furthermore, the version of GSI used in the ex- 
periments here includes the adjoint capability of Tremolet 
(2007, 2008). This adjoint differs from its previous in- 
carnation in Zhu and Gelaro (2008) in that it is not 
a line-by-line adjoint, but rather it is derived from 
a swap of operations used in the forward GSI. Com- 
bining the GSI adjoint with the GCM adjoint, GEOS-5 
has all the ingredients necessary to calculate the adjoint- 
based approximations to observation impacts discussed 
earlier (see also Gelaro et al. 2010). 

The main experiment period considered here is the 
same as that considered to obtain some of the results 
in DT09. Similarly to other related works, we study 
mainly the impact of observations on the 24-h forecasts. 
Twenty-four-hour forecasts and 30-h forecasts are issued 
for all 0000 and 1800 UTC times, respectively, for the 
month of August 2007, and second-order accurate ob- 
servation impacts are calculated over 0000 UTC times. 
The assimilation, forecasts and, when applicable, adjoint 
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integrations, are performed on a horizontal grid of 
2.5° X 2° resolution, with 72 hybrid vertical levels. 
Todling and Tremolet (2008) have shown that the rel- 
ative impact among the various observing systems is 
largely independent of resolution, rendering conclusions 
from the experiments here general. 

a. Role of norm 

1) State-space approach 

To illustrate the role played by the choice of weight 
matrices in the forecast error measures of the previous 
section, we consider three choices of norms when using 
the state-space approach. The first two are total energy 
measures. Following Errico et al. (2007), the inner 
product between two vectors, x 1 and x 2 , involving per- 
turbations in the zonal and meridional components of the 
wind, temperature, and surface pressure, is calculated 
using either one of the following expressions: 




X 


Uhk 

C RT 

/ / | / / i P rj~i/ rp/ . y f f 

u l u 2 + v l v 2 + T 1 T 2 + ~yrPslPs2 

1 r Pr 


J iJJc 

(37a) 


, = C t „x 2 = l z iJf 


X 


ilk 

u l u 2 + v l v 2 + 7jrT[T2 + —^PsiPa 
1 r Pr 


i,j,k 

(37b) 


where A H t j is a horizontal gridbox weight and the dis- 
tinction between the two norms is in how they weight 
the fields in the vertical, with Acr^- k and A Zijk being 
fractional weights, respectively, defined as 

A p 

A<r = (38a) 

'• Ps,i,j-P< 


A z 


i,i,k 


A[n P,jM 

ln Ps,i,r lnp ‘' 


(38b) 


For consistency with Errico et al. (2007), we refer to (37a) 
and (37b) as the ET-norm and EV-norm, respectively. 
The physical scaling coefficients c p = 1004.6 J kg -1 K -1 , 
R = 287.04 J kg” 1 K ~\T r = 280 K, and p r = 1000 hPa, 
are the specific heat at constant pressure, the gas con- 
stant of dry air, and a reference temperature and pressure, 
respectively. Detailed discussions of the applicability of 
these two flavors of the total energy norm appear in 
Lewis et al. (2001) and in Errico et al. (2007). Here, we 



Fig. 2. The fractional vertical weights Act (thin curve) and A z 
(thick curve) used for calculating the ET- and EV-norms, respec- 
tively. The dotted vertical line indicates the model levels. All cal- 
culated at a point where p s = 1000 hPa; the model-top pressure is 
0.01 hPa. [Similar to Fig. 1 of Errico et al. (2007)]. 


simply note that the energy in perturbations in the tro- 
posphere are emphasized when the fractional mass 
weights in (38a) are used; whereas the energy of per- 
turbations mainly concentrated in the midtroposphere 
and stratosphere are emphasized when the fractional 
distance weights in (38b) are used instead. This is illus- 
trated in Fig. 2 where the weights in (38) are displayed 
as a function of the 72 vertical levels of GEOS-5 for 
a grid point where p s = 1000 hPa. The thin curve is for 
the fractional mass weight (38a), and the thick curve is 
for the fractional distance weight (38b). In the former, 
the weights have a stepwise increase in the troposphere 
up to 300 hPa, above which level they decrease rapidly. 
The opposite happens in case of the latter fractional 
weights, where they are comparatively small in the tro- 
posphere and increase rapidly in the stratosphere and 
mesosphere. 

The works of LB04, E07, GZE07, and DT09 all use 
the ET-norm to calculate observation impacts. More- 
over, as mentioned in the introduction, these works also 
use a projection operator that is unit for all grid points 
roughly below 100 hPa, and zero above that. We have 
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compared state-space observation impacts using the ET- 
norm with and without this projection operator. The 
impacts change somewhat, but as expected, given that 
the ET-norm weights decay rapidly above the top of the 
projection operator box, 100 hPa, the differences in 
observation impact are not significant (not shown). In 
the present work we use a projection operator that only 
amounts to excluding the five top layers, mainly to avoid 
letting the EV-norm include anything too close to the 
model top, knowing that this changes essentially nothing 
when using the ET-norm. 

The third norm considered when calculating obser- 
vation impacts with the state-space approach follows 
from our considerations regarding the observation-space 
approach. That is, (6b) is evaluated for weighting matri- 
ces T k of the following form: 

09 ) 


Contrary to the ET- and EV-norms, this amounts to a 
time-dependent choice of weights. We have no indication 
that this affects in any significant way the conclusions 
drawn from the study that follows, and time dependency 
is consistent with the arguments of the previous section. 
This choice of norm is a simple attempt to have the state- 
space and the observation-space approaches use a simi- 
lar weighting factor, when the latter employs (34). 

Notice that using the weight matrices in (37) lead to 
observation impacts that have units of energy, while 
using the weights in (39) or, as we will consider later, 
those in (34), lead to nondimensional impacts. To fa- 
cilitate comparing the various norms and approaches 
we mostly show results as, nondimensional, fractional 
observation impacts. The fractional impact 5e%(/) of the 
ith observation set, in the measure m, is defined by 


8e™(i) = 100 X 


8e m {i) 


(40) 


where the summation runs over all observing sets in 
the figure under consideration. 

Figure 3 shows fractional observation impacts for 
each measure of interest and various observing systems: 
(Fig. 3a) the state-space ET-norm, (Fig. 3b) the state- 
space EV norm, (Fig. 3c) the state-space H T R -1 H weights 
(labeled R-norm), and (Fig. 3d) the observation-space 
R -1 weights (labeled R-omf). The breakdown of the 
observing system used during the period of the experi- 
ment is described in the figure caption. A preliminary 
version of the results in this figure appeared in Todling 
(2009). Let us concentrate first on Figs. 3a-c. These 
provide a direct assessment of the effect of the choice of 


norm in the “traditional” state-space observation im- 
pact calculation. Between Figs. 3a, b, the most striking 
difference shows in the role played by the radiosonde 
(RaobDsnd) and satellite observations: under the ET- 
norm radiosondes visibly have a comparable role to that 
played by the Advanced Microwave Sounding Unit- A 
(AMSU-A) and the Atmospheric Infrared Sounder 
(AIRS), whereas under the EV-norm AMSU-A domi- 
nates, while the role of radiosondes and AIRS becomes 
comparable. The fractional impact of the High Resolu- 
tion Infrared Radiation Sounder (HIRS) and the air- 
craft is flipped under these two norms, with the aircraft 
being significant in a largely tropospheric measure and 
HIRS becoming more dominate under a stratospheric 
measure. When the weights are based on the inverse of 
the observation error covariance matrix (Fig. 3c), the 
global fractional impacts seem to resemble more closely 
those obtained with the tropospheric ET-norm, though 
there are differences: aircraft observations show as be- 
ing slightly more significant than AMSU-A and AIRS, 
and almost comparable to radiosondes. 

Interestingly, the seemingly negligible fractional im- 
pact of AMSU-B when using either the ET- or EV- 
norms, is no longer so under the R-norm. This is at- 
tributed to the fact that both the ET- and EV-norms in 
(37) are dry measures and consequently provide a zero 
input gradient in specific humidity to the adjoint (fore- 
cast) sensitivity integration. The lack of any convective 
parameterization in the model adjoint results in the in- 
tegrated sensitivity to remain zero. Therefore, the tiny 
fractional impact appearing in Figs. 3a, b for AMSU-B 
are simply a result of how sensitive the moist channels are 
to changes in the temperature field within the analysis 
itself. On the other hand, under the R-norm, the input 
gradient to the forecast model adjoint is no longer zero in 
its specific humidity term; the model adjoint simply 
advects the initial gradient 24 h backward and gener- 
ates a nonzero forecast sensitivity in specific humidity 
that is then fed into the analysis adjoint; this, in turn, 
results in a nonnegligible contribution from the 
AMSU-B, moist-sensitive, channels (Fig. 3c). 

These three measures of observation impact on the 
24-h forecasts corroborate the, not surprising, signifi- 
cance of AMSU-A found elsewhere (LB04; E07; GZE07; 
Gelaro et al. 2010). Though we stress that the aim of 
the present work is not to evaluate and rank the ob- 
serving system, it is still instructive to look more closely 
at AMSU-A to see exactly where, for example, the dra- 
matic increase in fractional impact comes from when 
using the EV-norm. Figure 4 shows the breakdown of 
the fractional impacts for this instrument under the var- 
ious norms. Results include accumulated impacts from 
NOAA-15 , -76, and -18, as well as from Aqua . With the 
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Fig. 3. The fractional observation impact (%) for various instruments for all 31, 24-h forecasts of GEOS-5 valid at 
0000 UTC August 2007. The fractions are calculated with respect to total 31-day impact as defined in each case: (a) 
adjoint-based, total tropospheric energy norm; (b) adjoint-based, total stratospheric energy norm; (c) adjoint-based 
with forecast errors normalized with inverse of observation error covariance; and (d) OMF-based approach. Results 
from (a) and (b) first appeared in Todling (2009). The abbreviations along the vertical axis stand for observations of 
ships and buoy temperature, winds specific humidity, and near-surface pressure (Ship); cloud-drift winds (SatWind); 
Special Sensor Microwave Imager wind speeds (SSMIspd); Backscatter Ultraviolet Instrument total column ozone 
(SBUV2); radiosonde and dropsonde temperature, winds, specific humidity (RaobDsnd); Scatterometer winds (Qscat); 
Moderate Resolution Imaging Spectroradiometer (MODIS) clear-sky and water vapor winds; land observations 
of temperature, winds, surface pressure, and specific humidity (LandSfc); radiances from the High Resolution 
Infrared Radiation Sounder (HIRS) in (3) from NOAA-16 and -7 7; radiances from the Geostationary Operational 
Environmental Satellites (GOESND); aircraft temperature and winds (Aircraft); radiances from the Advanced 
Microwave Sounding Unit-A (AMSU-A) on the NOAA-15 , -76, and -18; Advanced Microwave Sounding Unit-B 
(ANSU-B) from NOAA-15 , -76, and -77, as well as on Aqua; and NASA Aqua , Atmospheric Infrared Sounders 
radiances (AIRS). 


EV-norms (Fig. 4b) the stratospheric channels 13 and 
14 have a very large fractional impact, and contribute 
considerably to the large accumulated fractional impact 
of AMSU-A seen in Fig. 3b. We should not be deceived 
by the seemingly small fractional impacts of the mid- 
tropospheric channels implied in Fig. 4b; the contribu- 
tion from these channels is still rather significant under 
this norm (see below). Examination of Fig. 4c, showing 
results when the R-norm is employed seem to indicate 
the use of the stratosphere-peaking channels to be 


significant, though results for the troposphere -peaking 
channels are comparable to when the ET-norm is used. 

To stress the point that when using the EV-norm the 
impact of channels peaking in the midtroposphere is 
similar to the impact seen by other norms, we show in 
Fig. 5 the fractional impact on the 24-h forecasts of 
channel 6 on NOAA-18 , summed over all 0000 UTC 
analyses of August 2007. Results for all measures are 
displayed in the figure. Neutral fractional impacts are 
shaded gray, with red indicating regions of positive 
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Fig. 4. As in Fig. 3, but for each of the 15 channels of AMSU-A on the NOAA-15, -16, -18 and on the NASA Aqua 
satellites; fractions are now calculated with respect to total 31-day impact of all AMSU-A channels for each case. 


impact and blue the regions where impact is negative. 
Figures 5a-c light up similar areas of the globe: the 
eastern Pacific Ocean is a particular area where all three 
measures agree about the significance of the observa- 
tions. There are also areas of disagreement, such as the 
southwestern Pacific Ocean, which the EV-norm indi- 
cates the observations to not be doing so well whereas 
results seem quite neutral when evaluated with either 
the ET- or R-norm. Clearly, there is a level of arbitrar- 
iness in what we choose to highlight in the plots, which is 
another way of saying that when comparing different 
measures of observation impacts there is considerable 
uncertainty involved until statistical significance is as- 
signed to the results (left for future work). 

2) Observation-space approach 

We now turn our attention to the observation-space 
approach and impact calculations based on (11) when 
using weighting factors defined by the inverse of the 
observation error covariance matrix, as in (34). Results 


of observation impacts on the 24-h forecast are seen 
in Figs. 3d, 4d, and 5d. 

Looking again at the summary plots in Fig. 3 and 
focusing on how the “traditional” state-space ET- 
normalized observation impacts (Fig. 3a) compare with 
the observation-space impacts (Fig. 3d), the most no- 
ticeable difference is in the fractional impact of radio- 
sondes. With the observation-space measure radiosondes 
seem much less significant, whereas aircraft, AMSU-A, 
and AIRS, seem to show roughly similar results in both 
approaches and norms, though their ranking changes. 
The reduced fractional impact of radiosondes measured 
with the observation-space approach is compensated for 
by the increased fractional impacts of AMSU-A and 
AIRS. A more direct comparison follows by contrasting 
Figs. 3c, d, since the norms in these two cases are closely 
related, and the comparison more faithfully addresses 
the difference in approaches rather than norms. How- 
ever, given the similarity between results with the ET- 
norm and those with the R-norm the same conclusions 
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Fig. 5. Fractional impacts on the 24-h forecasts from channel 6 of AMSU on NOAA-18 for the state-space approach using (a) the 
ET-norm, (b) the EV-norm, and (c) the R-norm, and (d) the observation-space approach using the R-norm. 


follow: the dominant observing systems under the 
observation-space measure are aircraft, AIRS, and 
AMSU- A. The reduced role of the radiosonde observa- 
tions can be understood by recalling that the observation- 
space-based measure compares how the 30- and 24-h 
forecasts fit the observations 30 and 24 h later, respec- 
tively. This measure does not explicitly incorporate the 
sensitivities of forecasts; it simply compares the quality 
of the forecasts directly. For the case of observation 
impacts on the 0000 UTC forecasts, the observation- 
space measure is saying that the forecasts from 1800 UTC 
are not as bad in predicting the radiosonde observations 
6 h later as one might be inclined to think. Taking the 
difference in (9) as a proxy for the difference in forecast 
quality, one would be led to think the 30- and 24-h fore- 
casts do not predict the radiosondes as well when evalu- 
ation is done with the state-space measures. Indeed, what 
this measure is saying is that there are larger sensitivities 
in the 30-h forecasts than in the 24-h forecasts when 
predicting the radiosonde observations (due to the con- 
siderable difference in the radiosonde network between 


0000 and 1800 UTC). Unfortunately, this is not infor- 
mation the cycling analysis system can make use of. 

Examining the results for the various channels of 
AMSU- A displayed in Fig. 4, we see the fractional im- 
pacts from the observation-space measure (Fig. 4d) re- 
semble more closely those obtained with the state-space 
approach when it uses the EV-norm (Fig. 4b). The 
stratospheric channels, 12-14, have a large role. Here 
again, we emphasize that this should not lead us to think 
that the role of the troposphere-peaking channels is not 
significant; it is simply that 6 h is enough time for the 
model stratosphere to reset its own errors, stressing the 
relevance of radiance assimilation at these levels. This 
is not unreasonable since there are plenty of other data 
in the troposphere and only those few channels in the 
stratosphere. Still the relevance of assimilating radiances 
in the troposphere is noticeable in the observation-space 
measure. This can be seen in Fig. 5d, where similar re- 
gions of significant impact from channel-6 brightness 
temperatures are highlighted when compared with other 
measures. 
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Fig. 6. Time series of total forecast error reduction calculated 
from the state-space approach in (6b), with the norm in (39) (red 
curve); the observation state approach using (11), with the weighting 
matrix in (34), and verified against the observations (blue curve) or 
against the analysis (green curve). 

b. Role of verification 

Let us now turn our attention to the role of the veri- 
fication in light of the discussion in section 2d. As we 
saw, the observation-space approach provides a way to 
examine the matter by explicitly calculating, for exam- 
ple, the first term on the right-hand side of (31) and the 
left-hand side of the same equation. Using the GEOS-5 
illustration considered here, Fig. 6 displays the time se- 
ries of total observation impacts, for each 0000 UTC 
analysis of August 2007 calculated in observation space 
using the weights in (34), but when the verification is 
taken to be (i) the observations [first term on right-hand 
side of (31), 8e y ; blue curve]; and (ii) the analyses con- 
verted to observation locations [left-hand side of (31), 
8e y=a \ green curve]. Note that the sum of all observation 
impacts used to calculate the fractional impacts Fig. 3d 
amounts to the sum of all impacts included in the time 
series displayed by the blue curve in the current figure. 
As a whole, it seems that the nearness of the blue and 
green curves is confirmation of the hope that it is rea- 
sonable to verify against the analysis, even at such short 
ranges as 1 day. Also, from the discussion in section 2d, 
the proximity of the curves is indication that the system 
is not so far from optimality — the last term in (31) does 
not appear to be very dominant — although there are 
exceptions, such as seen on day 29 of the time series in 
Fig. 6 2 

These conclusions, however, cannot be taken too far. 
Another, closer, look at the differences between veri- 
fying against the observations versus verifying against 
the analysis is presented in Fig. 7, where the total ob- 
servation impact for the 0000 UTC analyses of August 
2007 is partitioned into the different components of the 


2 It is important not to forget that here, as in most works, the 
expectation operator in expressions such as (31) is replaced with 
a simple summation (average) as only a single realization is avail- 
able in practice. 
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Fig. 7. Breakdown of observation impacts on the 24-h forecasts 
calculated with the observation-space approach. The blue bars 
show impacts when observations are used for verification; the 
magenta are for when the analyses are used for verification. A 
preliminary version of this result appeared in Todling (2009). 

observing system. The blue bars are for when verifying 
against the observations and the magenta bars are for 
when the projected analyses are used for verification. 
Verification against the analysis provides an overestimate 
of the impacts for each observing system. 

As pointed out in section 2d, similar calculations 
cannot easily be done in state space, but we can employ 
approximations such as those in (6). In practice, cal- 
culations from these approximations automatically in- 
volve verifying against the analysis. The red curve in 
Fig. 6 shows the time series of total impact per analysis 
time when the weights in (39) and the state-space-based 
second-order formula in (6b) are used. The state-space 
approach seems to overestimate the impact of the ob- 
servations in the 1-day forecasts beyond what one gets 
with the observation-space approach. Even when the 
norms in the two approaches are made to resemble each 
other, it is still the case that their corresponding errors 
live in largely different spaces resulting in considerably 
different total impacts. 

c. OMB-based observation impacts 

Though there are some clear similarities among re- 
sults obtained with the state- and observation-space ap- 
proaches and various norms, the truth is that the closer 
one looks, the more differences one finds. These dif- 
ferences become rather hard to reconcile with the idea 
that results from observation impact on the forecasts are 
to aid our ability to make improvements in the under- 
lying data assimilation system. As pointed out at the 
end of section 2, it is the opinion of this author that 
measures directly linked to the data assimilation cycle 
can naturally reduce some of the redundancy that 
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Fig. 8. Observations impacts on the 0-h forecast (analysis) calculated with the observation-space approach, when 
weighting is based on the inverse of the observation error covariances, (a) A summary similar to those appearing in 
Fig. 3; and (b) similar to those appearing in Fig. 4, and provides a breakdown of result for all platforms carrying 
AMSU-A instruments in August 2007. 


norm-dependent approaches bring. Fortunately, the by- 
products of any data assimilation system already provide 
most of what is needed to diagnose its own reliability, 
including how effectively various components of the 
observing system are used. The statistics of the time 
series of observation-minus-background (OMB) and 
observation-minus-analysis (OMA) residuals provide 
all needed information to allow such investigation to 
take place. This is not to say the task is easy. There is 
plenty of redundant information in the observing sys- 
tem (and therefore in the OMBs and OMAs), there are 
also observation and model biases that interfere with 
the robustness of these statistics, and there are spatial 
and temporal gaps in the observing system that make 
the problems quite arduous. What we know from esti- 
mation theory is that most of the information is found 
in these residuals. 

In this final section we then turn to the straightfor- 
ward calculation indicated in section 2e, resulting from 
considering the observation-space approach with m = 1. 
We construct so-called observation impacts from simply 
the differences of normalized squared OMA and OMB, 
as in (33). These are essentially the impact of observa- 
tions on the analyses (referred to below as “0 h” impact). 
The panels in Fig. 8 show results analogous to those in 
Figs. 3 and 4, but here the 0-h fractional impacts are 
evaluated in observation space. Concentrating first on 
the summary plot of Fig. 8a we see considerable re- 
semblance to the observation-space-based impacts on 
the 24-h forecasts shown in Fig. 3d. A noticeable differ- 
ence being that radiosondes and aircraft seem to play 
a relatively larger role affecting the analysis cycle than 
the 24-h forecasts; the roles of AMSU-A and AIRS seem 


to be more dominant in impacting the 24-h forecast as 
compared to radiosondes and aircraft. Turning to Fig. 8b, 
the breakdown of fractional impacts for AMSU-A shows 
a slightly dominant contribution coming from the largely 
tropospheric channels 5-9. When comparing with the 
24-h observation-space fractional impacts in Fig. 3d, 
the stratospheric channels show less impact affecting 
the cycling data assimilation than affecting the 24-h 
forecasts. This is consistent with the view that when set 
free the model tends to go back to the stratosphere it 
likes, while during the assimilation cycle the model does 
not have the opportunity to restore its stratospheric be- 
havior (e.g., Bloom et al. 1996). 

To provide further corroboration for the results 
above, an experiment covering the more recent period 
of December 2011 is briefly examined. The experiment 
now takes the analysis to be at its present full 0.625° X 
0.5° horizontal resolution, and uses a more recent ver- 
sion of GEOS-5 capable of handling the latest compo- 
nents of the observing system, such as refractivity derived 
from radio occultation observations from various plat- 
forms of the Global Positioning System (GPSRO), the 
Microwave Humidity Sounder (MHS), the Infrared At- 
mospheric Sounding Interferometer (IASI), and the Ad- 
vanced Scatterometer (ASCAT) on board the European 
Meteorological Operational ( MetOp-A ) satellite. As be- 
fore, observation impacts are only shown for 0000 UTC. 
The comparison that follows is not meant to be exhaus- 
tive. It is simply aimed at adding to the main points of 
the present contribution. 

Figure 9, is similar to Fig. 3, but now for December 
2011, and it shows summaries of observation impacts 
on the 24-h forecasts derived with the state-space 
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Fig. 9. (a) State-space, ET-norm, observation impact on the 24-h forecast and (b) observation-space, 
R-norm, observation impact on the 0-h forecast (analysis); as in (a) in Figs. 3 and 8, respectively, but for 
December 2011. 


approach using the ET-norm (Fig. 9a), and observation 
impacts on the analyses derived with the observation- 
space approach combining (33) with the weight in (34). 
To avoid being repetitive, results from other norms and 
approaches are not considered here. Caution should be 
exercised when comparing results from Figs. 3a and 9a. 
The first figure covers a summer month while the second 
covers a winter month. And, more importantly, some 
major components of the observing system change quite 
dramatically. Beyond the new instrument types already 
mentioned above, we also point out that NOAA-16 avail- 
able in August 2007 is no longer available in December 
2011, while NOAA-19 is only available during this latter 
period; AMSU-B has basically been replaced with MHS; 
and a decision was made not to use window channels of 
AMSU-A in this more recent version of GEOS-5. This 
is not a comprehensive list of changes in the observing 
system, but cites the most important for the present 
work. Even with all these differences, there are clear 
similarities between the two time periods in terms of 
which observing systems dominate. AMSU-A, radio- 
sondes, and aircrafts are still the ones impacting most 
the 24-h forecasts (see Fig. 9a). Next to them, AIRS is 
quite dominant, too, but now shares the role with IASI 
and the satellite winds; the lesser role played by the 
satellite winds in the August 2007 experiment might be 
associated with the considerably lower analysis reso- 
lution in that experiment. 

The most relevant comparison for the present work is 
made by looking at the two panels in Fig. 9. When it 
comes to influencing the analysis cycle (Fig. 9b), the role 
of AMSU-A is not as dramatic as indicated by the 24-h 
impact results (this is similar to what is found for August 
2007; see discussion of results for Fig. 8). However, in 
December 2011 the role of radiosonde is markedly larger 


than that of AMSU-A. This reduced impact of AMSU-A 
on the analysis is seen here as due to the addition of 
IASI and GPSRO to the present period. In a sense these 
new data types “steal” from the influence of AMSU-A; 
indeed, even AIRS, which in August 2007 was seen as 
a major contributor to the 0-h impact, has its role down- 
played now, likely due to the same reason. Such com- 
pensatory interplay among the various components of 
the observing system has been noticed in the observing 
system experiments of Gelaro and Zhu (2009). Compar- 
ing the 24- and 0-h impacts, we also see some ranking 
differences among the various observing systems. One of 
the most noticeable changes is seen for GPSRO: 24-h 
impacts lead us to think GPSRO contributes little, but the 
role of GPSRO in aiding the analyses is undeniable. This 
significant contribution of GPSRO to the analysis is 
consistent with results from the work of Cucurull (2010). 

Last, we look at the percentage of observations con- 
tributing positively to the observation impacts. These 
are calculated by simply counting the number of ob- 
servations whose impacts are negative (beneficial) for 
each instrument of interest, then dividing each result 
by the corresponding total number of observations for 
that instrument type, and multiplying the result by 100. 
Figure 10 shows results for both the state-space- and 
observation-space-based impacts on the 24- and 0-h 
forecasts, respectively, as displayed in Fig. 9. The panel 
on the right is analogous to the result shown by Gelaro 
et al. (2010, their Fig. 5), though for a different time 
period and observing system selection here. Gelaro et al. 
(2010) remind us that scalar theoretical analysis in ide- 
alized settings (M. Fisher 2006, personal communica- 
tion; Ehrendorfer 2007) indicate that roughly 60%-65% 
of the observations should contribute positively to the 
background when the accuracy of both backgrounds 
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Fig. 10. As in Fig. 9, but showing percentage of observations contributing positively to the (a) 24- and (b) 0-h 
forecasts. The vertical line in both panels indicates the 50% beneficial mark for reference. 


and observations are comparable. Results such as the 
one in Fig. 10a, suggesting that only roughly 50% of the 
observations lead to positive impact on the 24-h fore- 
cast, has led Gelaro et al. (2010) to the conclusion that 
operational systems cannot be expected to perform at 
theoretical levels. Results displayed in Fig. 10b state 
otherwise: operational systems do corroborate the scalar 
theoretical analysis. The bulk of the observing system 
contributes at exactly the expected theoretical range, 
that is, 60%-65% of the assimilated observations con- 
tribute positively. This is further corroboration of what 
was concluded while examining the result of Fig. 6, 
that is, global measures indicate that current operational 
data assimilation systems run near optimality. This is 
not to say further improvements are not needed or pos- 
sible. As results from Fig. 7 have indicated, there is still 
plenty of room for improvements when we start looking 
more closely. Similarly, results from Fig. 10 suggest work 
needs to be done to bring performance of some ob- 
serving systems to theoretical levels. 

4. Conclusions 

Studies of observation impact on the forecast have 
relied on the approach put forward by Langland and 
Baker (2004). A number of works have followed since. 
The present work is a contribution that provides insight 
on basic issues behind the technique. A few limitations 
and difficulties associated with the basic approach have 
been highlighted here, namely: (i) the need to rely on 
a norm not directly linked to the underlying data as- 
similation cycle; (ii) the need to rely on a verifying state; 
(iii) the need to rely on the model adjoint; and last, (iv) 
the added computational expense. Though (iii) has been 
tackled in the work of Liu and Kalnay (2008) by gen- 
eralizing the approach of Langland and Baker to work 


within the context of ensemble data assimilation pro- 
cedures, the other issues still remain. More complex is 
the idea of using observation impacts derived from these 
available techniques as an aid to improve on the use of 
observations in the corresponding cycling data assimi- 
lation system. 

The present work identifies two approaches to ob- 
servation impacts. The “traditional” method works in 
state space, while an alternative is to define measures 
of observation impact directly in observation space. 
Arguing that results from observation impact studies 
must be interpreted statistically, the present work re- 
casts the problem in the language of estimation theory. 
This allows studying more closely the assumptions in- 
volved in the methodology. In particular, it becomes 
clear that a state-space approach is more encompassing 
than an observation-space approach, simply because ob- 
servations span a smaller space than the full state space. 
But this advantage quickly disappears when realizing 
that the state-space approach requires a verification 
state normally not available in practice. Under certain 
conditions, the consequences of choosing the analysis 
for verification are investigated, showing explicitly how 
the corresponding observation impacts carry undesir- 
able correlations with the verification. The observation- 
space approach, on the other hand, allows verification to 
be made against the observations therefore, in principle, 
avoiding such undesirable correlations. Furthermore, the 
observation-space approach permits evaluating what is 
obtained when the analyses, instead of the observations, 
are used for verification. It is shown that only under op- 
timality can a system be indifferent to whether verifica- 
tion is done against the observations or the analyses. It 
is recognized in the present work that some of this ad- 
vantage disappears in practice since observations are 
usually bias corrected, thus making the observations also 
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correlated with the analysis. The effect of observation 
bias correction on impacts derived from observation- 
space approaches remains to be studied in future work. 

The fifth generation of the Goddard Earth Observing 
System (GEOS-5) is used to illustrate the main points of 
the theoretical part of the present work. First, the role of 
the norm used to defined the forecast aspect of interest 
is investigated. Results from the, usual, largely tropo- 
spheric measure are compared with those from when 
the norm weighs the vertical evenly. Observation-space 
considerations suggest investigating results from a third 
norm based on the inverse of the observation error co- 
variances projected onto state space. This is later compared 
with a similar norm, but directly defined in observa- 
tion space. Observation impacts on the 24-h forecasts 
obtained with these different norms show considerable 
similarities, but also show differences that might be dif- 
ficult to sort out if results are to be used to try to revisit 
the observing system used in the data assimilation sys- 
tem to try to improve its cycled performance. 

Second, GEOS-5 is also used to investigate the role of 
the verification, and in particular the effect of using the 
analysis for verification. Within the observation-space 
approach, using the analysis instead of the observations 
for verification seems to make only a small difference, 
at least when it comes to the impact of the entire ob- 
serving system on the 24-h forecasts. From the theoret- 
ical discussion, this result is interpreted as indicative 
of the assimilation system operating near optimality. 
However, it is also shown that this interpretation cannot 
be pushed too far. A closer look at the observation im- 
pact for the individual components of the observing 
system indicates impacts tend to be overestimated when 
verification is done against the analysis. Moreover, when 
the state-space approach uses a similar norm to calculate 
observation impacts on the 24-h forecasts, the impacts 
are an even larger overestimate than when derived in 
observation space. 

Ultimately, an argument is made that observation 
impacts might as well be defined on the basis of the 
time series of the readily available observation-minus- 
background and observation-minus-analysis residuals. 
The argument follows directly from theoretical consid- 
erations. The main advantage of using these residuals to 
define observation impacts is that whatever is learned 
from them is immediately related to the cycling data 
assimilation system. They reflect how observations af- 
fect the analyses. They relate directly to the traditional 
use of residuals going back to the works of Kailath (1968), 
and largely explored by many works in the meteoro- 
logical and oceanic data assimilation literature. It is 
even shown that our present data assimilation systems 
corroborate back-of-the-envelope calculations for how 


much the assimilation of observations should in prin- 
ciple, positively, impact the backgrounds. 

The observation-space approach definition of obser- 
vation impact used in the present work, namely the 
difference between an observation-space error mea- 
sure calculated for two consecutive analyses, was simply 
employed for convenience of the mathematical analysis 
and to facilitate comparison with the state-space ap- 
proach. Evaluation of the impact of observations through 
methods based on information content and degrees 
of freedom for signal should be just as well suitable 
(e.g., Chapnik et al. 2006; Lupu et al. 2011). The main 
point being that observation-space approaches rely- 
ing on observation-minus-background and observation- 
minus-analysis residuals are the only ones directly 
relating the effectiveness of the observations to the 
quality of the data assimilation cycle. 
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APPENDIX A 

Proof of Equation (21) 

In this appendix we derive expression (21), for linear 
dynamics, making use of the typical assumption of time- 
uncorr elated model errors. We start by making use of 
the following recursion: 

m — 1 

e k\k-m ~ ^k,k-m e k-m\k-m X ^k,k-j e k-j’ C^-) 

expressing the forecast error at time t k initialized from 
an analysis obtained at time 6c- m, where e q k represents 
the /r-vector model error. Consequently, the expected 
value of the outer product of this error vector with itself 
defines the forecast error covariance: 
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where the first equality is obtained by noticing that the 
analysis error at time t k - m is uncorrelated with model 
errors in its future, that is, for t k > t k - m , and the second 
equality makes use of the fact that model errors are 


uncorrelated in time and assumed to have covariance 

Q* = <4(4) t >- 

Therefore, the forecast error difference \P k 
becomes, 


AP; 


= of 




k\k-m+l k\k—m 

(p a 

k,k-m+l^ r k-m+l\k-m+l 


Q, 


k-m+l)^k,k-m+l 


M 


P a M 1 

k,k~m k—m\k—m k,k—m 


^ k,k-m+l^k-m+l\k-m+l 

= M (P a 

iyi k,k-m+l^ r k-m+l\k-m+l 


Q.- m+ i)M T 


'k,k—m + 1 

^k-m+l\k-m)^k,k-m+l ’ 


M M P a M t 

m k,k-m + l m k-m+l,k-m r k-m\k-m m k-m + l,k-m m k,k-m+l 


(A3) 


where the last equality is obtained after application of 
the usual one-time forecast error covariance propaga- 
tion expression: 

p/ - M o a |\/| T 

r k-m+l\k-m ™'k-m+l,k-m r k-m\k-m' }l 'k-m+l,k-m 

+ Q k-m+ 1’ ( A4 ) 

at tk-m+i- This completes proof of (21). 

APPENDIX B 

Proof of Equation (28) 

To derive an expression relating the perceived 
and actual forecast error changes notice first that by 


adding and subtracting the true state at time t k from 
(5), the expected perceived error can decomposed 
into 

(4\t) = ((4\e - 4) T T *(4 - 4)) = Tr[T,(P{„ + PD] 

- 2 <(4) T Vi>, (Bl) 

where e v k = x v k — x l h is the error in the verification xj£, 
and = {e k (e v k ) T ) is its corresponding error covariance. 
Inserting this in expression (9), the expected perceived 
error change becomes 


(del) = TWP {\k-m + i + K)} ~ 2((4) T T,4| fc - m+ i) - Tr[T,(P{„. 

= ^[T^AP^] ~ 2((4) 1yfc( e {|A:-m+l ~ e k\k-m )) 

— i^ e k) 2(e<;T k^k,k-m+l^k-m+l^k-m+l\k-m) 

= ( Se k ) ~ 2Tr[kl_ m+1 Mj yfc _ m+1 T yt (e^dJ_ m+1 | fc _ m )], 


PD] 


■2((eD T T 


k e k\k-m 


—m) 


(B2) 


where the equality before last is obtained after using 
(15) and (17), and we remind the reader again that 
though the notation i \ j resembles the estimation 
theory notation for conditional means, the use here 
is symbolic — this is the reason why the matrices in 
the expression above can be moved outside of the 
expectation operation ( • ) . This completes the proof 
of (28). 


APPENDIX C 

Proof of Equation (29) 

When the verification is chosen to be the underlying 
analysis, that is, x v k = x^, the relationship between the 
expected perceived and actual errors derived in appen- 
dix B can be explored farther. In this case, the error 
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cross-covariance matrix inside the trace term on the right- 
hand side of (28) involves the analysis error e v k = e^ k . 
To evaluate this cross covariance we notice first that 
the residual vector d k \i can be approximated to first 
order as 


e k\k ™'k,k—m+l e k—m+l\k—m+l 
m— 2 

+ I M kk _ j (K k _ j d k _. k _ j _ 1 -el_ j ), (C2) 

7=0 


d k\e = y k- = 4 + •»*(**) - 

^ 4~^k\fk\V (Cl) 

Furthermore, assuming the forecast error propa- 
gates nearly linearly as in the propagation of analy- 
sis error from time t k - m+1 to time t k follows the 
recursion: 


and relates the analysis error at time t k with the analysis 
error from time t k - m+1 all the way to time t k , plus a term 
that amounts to the propagation of analysis increments 
from all times between times t k - m+ 2 to t k , and the prop- 
agated contribution from all model error components, 
e q k _j for j = 0, . . . , m - 2. This recursion is applicable 
within the validity of piecewise linearizations of the vari- 
ous time intervals between the times t k - m + i and t k . 

Substituting (C2) in the cross-covariance term 
in (28), and using that e a k _ m+Mk _ m+l = s{_ m+1|fe _ m + 
^k— m+l^k— m+l\k— ra? if follows that 


v=a jT 

L k u k—m+l\k—m 


[M 


k,k— m+l^k— m+l\k— m ^k,k — m + 1 ^k— m+l^k— m+l,k— m 


m — 2 


^ ^k,k-j(y^k-flk-j,k-j - 1 e k-p^k-m+l\k-m 

7=0 

k,k—m+ 1( ^k— m+l\k— m^k— m + l\k— m ^k— m+1^ k— m+l) 

m— 2 

S' ^k,k-j ^k-j y^k-jjc-j - 1 ^k-m+1 \k-m 


7=0 

m—2 


^k,k—m+l^^ L k—j^'k—m+l 


S ^k,k-j^k-j\^k-j, k -j-l^k-m+l\k-m 


(C3) 


for observations y° k = h^(x^) + e ° h , and noticing that the 
observation error e° h is uncorrelated with the model error 
for all times t k and t £ , as well as it is uncorrelated with 
all forecast errors calculated for forecasts issue for 
t < k. This concludes the proof of (29). 
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